Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache creation of states_token_mapping in the __init__ of RegexFSM #492

Merged
merged 4 commits into from
Jan 11, 2024

Conversation

RobinPicard
Copy link
Contributor

PR aimed at talking about the resolution of issue #483

I tried to do it in a way that requires as few modification to existing functions as possible. It implies modifying the type of several variables for them to be cacheable/json-serializeable. It looks a bit awkward and leads to the initialization of the RegexFSM instance taking a bit more time when not cached because of the type conversions, but I thought that could be the start of further discussions on the topic

@rlouf
Copy link
Member

rlouf commented Dec 30, 2023

Great! I wonder if using diskcache everywhere in the codebase instead of perscache could help make the code cleaner? See #441

@RobinPicard
Copy link
Contributor Author

RobinPicard commented Dec 30, 2023

I think it makes the code much cleaner in that case (and does not change the use of @cache in the OpenAI model)

@RobinPicard RobinPicard marked this pull request as ready for review December 31, 2023 08:15
@rlouf rlouf mentioned this pull request Jan 6, 2024
@lapp0
Copy link
Contributor

lapp0 commented Jan 7, 2024

Good work!

@RobinPicard RobinPicard requested a review from rlouf January 10, 2024 20:17
@rlouf rlouf force-pushed the cache_regexfsm_init branch from 542d30b to 6017d30 Compare January 11, 2024 11:50
@rlouf
Copy link
Member

rlouf commented Jan 11, 2024

Thank you!

@rlouf rlouf merged commit 076bd98 into dottxt-ai:main Jan 11, 2024
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants